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is treated as a matrix. The general term for this process is "crosstalk correction", since 
it removes the information from a source that has fallen into a channel adjacent of the 
correct channel. This information spillover is analogous to crosstalk in bundled 
telephone lines, which results in a listener hearing the conversation from another line. 

This set of equations must be solved for each pixel location. It is essential, 
then, that the images from all of the channels be precisely aligned with one another so 
that the correct measurements are entered into the equations for each pixel location. 
Therefore, a computational process is applied to the images belonging to a field of view 
before the crosstalk correction is applied. The "field of view" is the scene captured in 
an image. The images belonging to a field of view may be referred to as the "image 
ensemble" for that field of view. 

FIGURE 24 is a block diagram of the operations required to carry out the 
spatial and spectral corrections. It includes two significant image processing 
operations, applying spatial corrections executed in a block 305 and applying spectral 
cross talk corrections, executed in a block 306. In block 305, the images from a 
plurality of channels are spatially aligning, and in block 306, corrections are applied to 
remove the channel-to-channel crosstalk. There are two classes of information that 
support the operations executed in blocks 305 and 306. The first is the class of 
calibration constants, derived from on-line calibration of the instrument during its 
operation. The second class of supporting information is that for which stored tables of 
constants are accessed during operation, but not modified. 

In a block 301, a calibration image that is generated from a plurality of offset 
stripes (the spatial offset for the calibration image being known) is introduced to the 
instrument. The instrument then computes the spatial offsets between the stripes in a 
block 302. The X,Y (horizontal and vertical) spatial offsets are determined in a 
block 303. Note that the image misalignment responsible for such spatial offsets may be 
subject to thermal drift and other factors that may vary during operation. The X,Y 
spatial offsets are used in block 305, along with other information, to apply spatial 
corrections. 

The stored constants will have been derived from measurements and from the 
known characteristics of the objects to be imaged. Different stored constraints will be 
applied to the spatial corrections executed in block 305 than are applied to the crosstalk 
corrections executed in block 306. For example, the general positions of the channels 
relative to each other known, and these positions are required for the spatial corrections 
executed in block 305. The stored data relating to the general positions of the channels 
relative to each other are provided to a processor executing the spatial corrections of 
block 305 in a block 308. With respect to the crosstalk corrections executed in 
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block 306, stored data relating to the inverted source coefficients are provided in a 
block 309. It should be noted that additional stored measurements and known 
characteristics can also be incorporated into the second class, and provided to the spatial 
correction operation or the crosstalk correction operation, as appropriate. 

The image processing stages can be applied in real-time as images are collected 
during system operation, or by offline access to stored image files. The images to be 
spatially and spectrally corrected in blocks 305 and 306 are provided by a block 304. 
The resulting corrected images are output in a block 307, and are suitable for further 
processing and analysis, free of the information degradation caused by crosstalk and 
misalignment. 

A block diagram of the general operations of the preferred embodiment of the 
present invention is shown in FIGURE 25. An "Image Ensemble" refers to a set of 
images, all of which depict the same field of view, but each of which has been 
constructed of signals from a particular channel. The number of images in an ensemble 
is equal to the number of channels in the instrument. The image ensemble is generated 
in a block 401. 

The X (horizontal) and Y (vertical) offsets are required in order for the 
alignment process to operate on the image ensembles. These offsets are provided in a 
block 403. As discussed above with respect to FIGURE 24, calibration images are 
processed to determine these offset values. The calibration process may be run as a 
preliminary step in the operation of the instrument. The calibration may be repeated 
periodically to track drift in the image registration caused, for example, by changes in 
temperature. The process of generating the offsets is illustrated by a block diagram in 
FIGURE 26. 

In a block 404, images are aligned by shifting rows by pixel increments. In a 
block 406, an aligned image is generated by a floating point interpolation in a, using 
interpolation kernels generated in a block 405, based on the X,Y offsets determined in 
block 403. In a block 407, the crosstalk correction process described above is applied, 
using a spectral coefficient matrix provided by a block 408. Then in a block 409 the 
8-bit grayscale is restored, resulting in a final image ready to output in a block 402. 

Referring now to the block diagram of FIGURE 26, for each image, represent 
by a block 501, a first operation in the generation of the spatial offsets is that of 
boundary detection, executed in blocks 502 and 503. From the boundary information 
determined in block 502, a Fourier Transform is applied in a block 504, and then a Fast 
Fourier Transform is applied in the reference channel From the boundary information 
determined in block 503, a Fourier Transform is applied in a block 505, and then a Fast 
Fourier Transform is applied in the data channel. The results of both Fast Fourier 
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Transforms are multiplies in a block 508, and an Inverse Fourier Transform is applied 
to the result in a block 509. Then accumulated 2D correlograms (1 per data channel) 
are generated in a block 510. The process is repeated for additional images. A detailed 
description of the process illustrated in the block diagram of FIGURE 26 is provided 
below. Note that FIGURE 27 illustrates exemplary images before and after correction. 

A two-dimensional gradient operator is used to suppress flat surfaces and to 
enhance object boundaries. The operator builds signal in regions where the slope of the 
intensity is large in both the vertical and the horizontal directions. The linear 
expression for this operator is as follows: 

G = — — . 

dy dx 

Object boundaries carry all of the energy in the images transformed by the 
gradient operator. What is needed next is a way to overlay each data image onto the 
reference image and measure how much vertical shift and horizontal shift is required to 
line up the object boundaries. Cross-correlation is a preferred method for measuring 
these shifts, or offsets, between images. 

The cross-correlation of two signals is comprised of convolving the two signals 
and extracting information from the output of the convolution operation, which is called 
the correlogram. The operation of convolution in the time domain is defined by the 
following equation: 

f 1 (t)®f 2 (t)= ZfiQtf 2 (t-X)dX. 

The value of the convolution for every time sample, t, is the sum over infinity of 
the product of the two functions, but with the second function offset by time t. The time 
delay between the two signals can be determined by finding the peak in the correlogram. 

For image realignment, the convolution is applied in two dimensions to 
functions in x,y space, as follows: 

f i (x, y) ® f 2 (x, y) = jjf, ( £ ,n)f 2 (x - e, y - r|)dedr| . 

The alignment method is based on the premise that there will be similar 
structures in the images to be aligned. In the ideal case, the second image is identical to 
the first except for a shift in x and y. Furthermore, the shift can be represented as the 
convolution of the function with the two-dimensional Dirac delta function, thus: 
fi(x,y) = fi(x-x Q ,y-y Q ), 
and 

f 2 ( x > y) = f 1 (x,y)<8>S(x-x 0 ,y-y Q ). 
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